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(a) selecting a frame cluster in said inpii^ideo sequence which 
corresponds to a most static one o^^d video segments; 

(b) computing a content valuej^said selected frame cluster; 

(c) using said computed^iemitent value to cluster remaining frames 
in said input vuJjto sequence. 

The method of claim 1, wherein/in said (a) said frame cluster is 
using a refined feature space representation of said input video sequence. 
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3. The method of claim 1, wherein/ in said (a) each of said pluraUty of 
frames is transformed into a histogram vector |indicative of a spatial distribution of 
colors in said each of said plurality of frames. 

4. The method of claim 3, wherein in said (a) each of said plurality of 
frames is divided into a plurality of blocks,|each of said plurality of blocks being 
represented by a histogram in a color spaqp indicative of a distribution of colors 
within each of said plurality of blocks. 

5. The method of claim 3, whferein each of said plurality of frames is 
divided into a plurality of blocks and each said histogram vector comprises a plxirality 
of histograms in a color space, each of sa|d plurality of histograms corresponding to 
one of said plurality of blocks. 

6. The method of claim / 2, wherein said refined feature space 
representation is obtained using a singi/lar value decomposition of said input video 
sequence. 



1 7. The method of claim 6/ wherein said singular value decomposition is 

2 performed using frames selected with a fixed interval from said input video sequence. 
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8. 



The method of claim 



a feature frame matrix, and wherein 
on said feature frame matrix. 



, wherein said selected frames are arranged into 
said singular value decomposition is performed 
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1 9. The method of claim 6, wherein said singular value decomposition 

2 produces a matrix, each column of said matri^ representing a frame in a refined 

3 feature space corresponding to a frame in said inmit video sequence. 

1 10. The method of claim 1, furtheo comprising (d) using said clustered 

2 frames to output a motion video representatwe of a summary of said input video 

3 sequence. / 

1 11. The method of claim 1 , fiirther comprising (d) outputting a plurality of 

2 keyframes, each of said plurality of keyframes representative of said clustered frames. 

1 12. The method of claim 2, wl^rein said selecting comprises locating a 

2 cluster closest to an origin of said refined feature space. 

1 13. The method of claim 2, wherein said (c) comprises: 

2 (c)(1) sorting a plurality of vectors in said refined feature space in 

3 ascending order according to a distance of each of said vectors 

4 to an origin of said refined feature space representation; 

5 (c)(2) selecting a vecto^ among said sorted vectors which is closest to 

i 

6 an origin of said refined feature space representation and 

7 including said selected vector into a first cluster; 

8 (c)(3) clustering said plurality of sorted vectors in said refined feature 

9 into a pluralit>|of clusters according to a distance between each 

10 of said plurality of sorted vectors and vectors in each of said 

1 1 plurality of clusters and an amount of information in each of 

1 2 said plurality/of clusters. 

1 14. The method of claim 13, wherein in said (c)(3) said plurality of sorted 

2 vectors are clustered into said /plurality of clusters such that said amount of 

3 information in each of said plurality of clusters does not exceed an amount of 

4 information in said first cluster. 
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1 15. The method of claim 13, whereini said first cluster is composed of 

2 frames based on a distance variation between sd^d fi"ames and an average distance 

3 between fi-ames in said first cluster. 

1 16. The method of claim 13, whereili each of said plurality of clusters is 

2 composed of fi"ames based on a distance variation between said fi"ames and an 

3 average distance between fi-ames in said each of said plurality of clusters. 

1 17. A method for summarizing a (jontent of an input video sequence, said 

2 method comprising: 

3 (a) selecting fi"ames fi'omf said input video sequence, said selected 

4 firames being taken aua fixed interval; 

5 (b) creating a feature firapie matrix using said selected frames; 

6 (c) performing a singmar value decomposition on said feature 

7 frame matrix to pbtain a matrix representing said video 

8 sequence in a refinpd feature space; 

9 (d) selecting a cluste/ in said refined feature space corresponding 

10 to a most static vmeo segment; 

11 (e) computing a ccmtent value corresponding to said selected 

12 cluster; | 

13 (f) using said computed content value to cluster frames in said 

14 input video sequence. 

1 18. A method for segmenting an input vifleo sequence, said input video 

2 sequence comprising a plurality of frames, said murality of frames being grouped 

3 into a plurality of video shots, said mraiodcomgnsing: 

4 (a) computing a simil^ji^etween each of said plurality of frames 

5 and a frame preced|ng^id each of said plurality of frames in 

6 time; 

7 (b) segmenting said in^ut video sequence into said plurality of 

8 video shots accordpg to said computed similarity. 
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1 19. The method of claim 18, wherein laid similarity is calculated using a 

2 refined feature space representation of said input video sequence. 

1 20. The method of claim 19, wherein said refined feature space 

2 representation is created using a singular value decomposition of said input video 

3 sequence. / 

1 21. The method of claim 20, whermn said singular value decomposition is 

2 performed using frames selected with a fixed mterval from said input video sequence. 

1 22. The method of claim 21, wMerein said selected frames are arranged 

2 into a feature frame matrix, and wherein said singular value decomposition is 

3 performed on said feature frame ma^^. ] / 

1 23. The method of cmimi^STv wherein said performed singular value 

2 decomposition produces a matrix, eacm ccmamn of said produced matrix comprising a 

3 frame in said refined feature space rfpfe^nting a frame in said input video sequence. 

1 24. The method of claim is', further comprising (c) extracting features 

2 from eachof said plurality of video shots. 

3 25. A method for determiiUng a similarity between a first and a second 

4 frame in an input video sequence, saidlmethod comprising: 

5 (a) calculating a refined feature space representation of said input 

6 video sequencj; 

7 (b) using said calculated representation to compute said similarity 

8 between said first and said second frames, 

9 26. The method of claim 25, wherein in said (a) said refined feature space 
10 representation is calculated using aJsingular value decomposition. 
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1 27. The method of claim 18, wherpn in said (b) said computed similarity 

2 is compared to at least a first threshold siirJlarity and a second threshold similarity, 

3 and said input video sequence is segmentecjf according to a result of said comparison. 

1 28. The method of claim 1^, wherein if in said (b) said computed 

2 similarity is below a first threshold siimlarity, said each of said plurality of frames is 

3 put into a one of said plurality 9^wle0 shots containing said precedent in time firame. 



4 29. The method [of 

5 similarity is above a second thre 

6 is designated as a shot bounda 



18, wherein if in said (b) said computed 
lilarity, said each of said plurality of frames 



7 30. The method of daim 18, wherein if in said (b) said computed 

8 similarity is between a first threshold similarity and a second threshold similarity, 

9 said each of said plurality of fi-ames is put into a one of said plurality of video shots 

10 according to a further analysis performed using additional frames from said plurality 

1 1 of fi-ames. 

A computer-readable medium containing a program for summarizing a 
cjpnf^nt of an input video sequence, said inputmdeo sequence comprising a plurality 
of firames, said plurality of frames being groi/ped into a plurality of video segments, 
said program comprising: 

(a) selecting a frame cluster in said input video sequence which 
corresponds to a most static video segment; 

(b) computing content value in said selected frame cluster; 

(c) using said computed content value to cluster remaining frames 
in said input video sequence. 




1 32. The computer-readable /medium of claim 31, wherein in said (a) said 

2 frame cluster is selected using a refped feature space representation of said input 

3 video sequence. 
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1 33. The computer-readable medium of claim 31, wherein in said (a) each 

2 of said plurality of frames is transformed/into a histogram vector indicative of a 

3 spatial distribution of colors in said each of^aid plurality of frames. 

1 34. The computer-readable meaiiun of claim 33, wherein in said (a) each 

2 of said pluraUty of frames is divided intola plurality of blocks, each of said plurality 

3 of blocks being represented by a histogram in a color space indicative of a 

4 distribution of colors within each of said /plurality of blocks. 

I 

1 35. The computer-readable jnedium of claim 33, wherein each of said 

2 plurality of frames is divided into a plurality of blocks and each said histogram vector 

3 comprises a plurality of histograms |in a color space, each of said plurality of 

4 histograms corresponding to one of said plurality of blocks. 

1 36. The computer-readable medium of claim 32, wherein said refined 

2 feature space representation is obtained using a singular value decomposition of said 

3 input video sequence. | 

1 37. The computer-readable medium of claim 36, wherein said singular 

2 value decomposition is performed using frames selected with a fixed interval from 

3 said input video sequence. | 

1 38. The computer-readable medium of claim 37, wherein said selected 

I 

2 frames are arranged into a featije frame matrix, and wherein said singular value 

1 39. The computer-readable medium of claim 33, wherein said singular 

2 value decomposition produces a matrix, each column of said matrix representing a 

3 frame in a refined feature space corresponding to a frame in said input video 

4 sequence. 
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40. The computer-readable medium of claim 31, further comprising (d) 
using said clustered frames to output a video representative of a summary of said 
input video sequence. 

41. The computer-readable medium of claim 31, further comprising (d) 
outputting a plurality of keyframes, eachfof said plurality of keyframes representative 
of said clustered frames. | 

42. The computer-readable 
comprises locating a cluster closest 



medium of claim 32, wherein said selecting 
to an origin of said refined feature space. 



43. 

comprises: 



The computer-readable medimn of claim 32, wherein said (c) 



(1) 



(2) 



(3) 



sorting a pluraMty of vectors in said refined feature space in 
ascending order according to a distance of each of said vectors 
to an origin of said refined feature space; 
selecting a vector among said sorted vectors which is closest to 
an origin of said refined feature space and including said 
selected vectomnto a first cluster; 

clustering saidlplurality of sorted vectors in said refined feature 
into a pluralit| of clusters according to a distance between each 
of said plurality of sorted vectors and each of said plurality of 
clusters and amount of information in each of said plurality 
of clusters. 



44. The computer-readable medium of claim 38, wherein in said (3) said 
pluraUty of sorted vectors are clustered into said plurality of clusters such that said 
amount of information in each of sj id plurality of clusters does not exceed an amount 
of information in said first cluster. 



V 
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1 45. The computer-readable medimii of claim 38, wherein said first cluster 

2 is composed of frames based on a distance yariation between said frames and said 

3 first cluster. 
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1 46. The computer-readable mecMum of claim 38, wherein each of said 

2 plurality of clusters is composed of frames ibased on a distance variation between said 

3 frames and said each of said plurality of clfisters. 

1 47. A computer-readable medlim containing a program for summarizing a 

2 content of an input video sequence, said program comprising: 

3 (a) selecting frames pith a fixed interval from said input video 

4 sequence; 

5 (b) creating a featur^ frame matrix using said selected frames; 

6 (c) performing a s^gular value decomposition on said feature 

7 frame matrix to/obtain matrix representing said video sequence 

8 in refined featuf e space; 

9 (d) selecting a cluster in said refined feature space corresponding 

10 to a most staticj video segment; 

11 (e) computing a jcontent value corresponding to said selected 

12 cluster; 

13 (f) using said ccfmputed content value to cluster frames in said 

14 input video sequence. 

1 48. A computer-readable medium conifaining a program for segmenting an 

2 input video sequence, said input video sequencer comprising a plurality of frames, said 

3 plurality of frames being grouped info ajmurality of video shots, said program 

4 comprising: 

5 (a) computing a siAil^fri^jbetween each of said plurality of frames 

6 and a subsequentlii'^me frame; 

7 (b) segmenting said input video sequence into a plurality of shots 

8 according to said dbmputed similarity. 
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1 49. The computer-readable medium of claim 18, wherein said similarity is 

2 calculated using a refined feature space representation of said input video sequence. 

1 50. The computer-readable medium of claim 19, wherein said refined 

/ 

2 feature space representation is created usipg a singular value decomposition of said 

3 input video sequence. 

1 51. The computer-readable niedium of claim 20, wherein said singular 

2 value decomposition is performed using frames selected with a fixed interval fi-om 

3 said input video sequence. | 

1 52. The computer-readable /medium of claim 21, wherein said selected 

2 fi-ames are arranged into a feahfre mme matrix, and wherein said singular value 

3 decomposition is performed on said f ut ure frame matrix. 

1 53. The computer-readableEnedium of claim 22, wherein said performed 

2 singular value decomposition produces a matrix, each column of said produced 

3 matrix comprising a frame in said refined feature space representing a frame in said 

4 input video sequence. 

1 54. The computer-readable medium of claim 18, wherein said program 

2 further comprises (c) extracting features from each of said plurality of video shots. 

3 55. A computer-readable medium containing a program for determining a 

4 similarity between a first and ?! second frames in an input video sequence, said 

5 program comprising: | 

6 (a) calculating a refined feature space representation of said input video 

7 sequence; and 

8 (b) using said calcuh ted representation to compute said similarity between 

9 said first and said second frames. 
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10 56. The computer-readable mediifai of claim 25, wherein in said (a) said 

11 refined feature space representation is/ calculated using a singular value 

12 decomposition. 

1 57. The computer-readable medium of claim 18, wherein in said (b) said 

2 computed similarity is compared to at least two threshold similarities, and said input 

3 video sequence is segmented according tq a result of said comparison. 

1 58. The computer-readable n|bdium of claim 48, wherein if in said (b) said 

2 computed similarity is below a first threshold similarity, said each of said plurality of 

3 fi-ames is put into a one of said plurality of video shots containing said precedent in 

4 time fi-ame. 
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59. The computer-reaflabldmedium of claim 48, wherein if in said (b) said 
computed similarity is above a seco^fereshold similarity, said each of said plurality 
of firames is designated as a shot bo/inc 

60. The computer-readablelmedium of claim 48, wherein if in said (b) said 
computed similarity is between a first threshold similarity and a second threshold 
similarity, said each of said plurality/ of fi"ames is put into a one of said plurality of 
video shots according to a further analysis performed using additional fi"ames fi-om 
said plurality of fi'ames. 

61. The method of claim 18, further comprising (c) extracting features 



fi"om each of said plurality of vide< 
said plurality of video shots. 



shots and using said extracted featiu-es to index 



1 62. The method of claiiA 61, wherein said extracted features are features of 

2 a video frame representative of saip each of said plurality of video shots. 



1 63. The computer-readable medium of claim 48, wherein said program 

2 further comprises (c) extracting features from each of said plurality of video shots and 

3 using said extracted features to ir^ex said plurality of video shots. 
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1 64. The method of claim 63, wherem said extracted features are features of 

2 a video frame representative of said each of said plurahty of video shots. 

1 65. A method of calculating a degree of visual changes in a video shot, 

2 said video shot comprising a plurality of frames, said method comprising: 

3 (a) performing a singular value decomposition on said plurality of frames, 

4 wherein said singular value! decomposition produces a matrix, each 

5 column of said matrix representing a frame in a refined feature space 

6 corresponding to a frame in said plurality of frames; 

7 (b) using said matrix to calculate said degree of visual changes in said 

8 video shot. I 

1 66. The method of claim 65, wherein said (b) comprises calculating said 

/ j\ /rank(A) 

2 degree of visual changes in saia vi&eofpot as a sum 1 ^ , wherein Vy are 

3 elements of said matrix. ^ | I J 

1 67. A computer-readable medium containing a program for calculating a 

2 degree of visual changes in a video shot, said video shot comprising a plurality of 

3 frames, said program comprising: | 

4 (a) performing a singular vplue decomposition on said plurality of frames, 

5 wherein said singular jvalue decomposition produces a matrix, each 

6 column of said matrix Irepresenting a frame in a refined feature space 

7 corresponding to a frame in said plurality of frames; 

8 (b) using said matrix to Calculate said degree of visual changes in said 

9 video shot. | 

1 68. The computer-readable medium of claim 67, wherein said (b) 

2 comprises calculating said degree! of visual changes in said video shot as a sum 

|rank(A) ~ I 

3 ^1 ^ Vi ' wherein Vy are elements of said matrix. 

f 
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1 69. A method of calculating an evenness of color distributions in a video 

2 shot, said video shot comprising a plurality of fi^es, said method comprising: 

3 (a) performing a singular value decomposition on said plurality of frames, 

4 wherein said singular value decomposition produces a matrix, each 

5 column of said matrix represmting a frame in a refined feature space 

6 corresponding to a frame in said plurality of frames; 

7 (b) using said matrix to calcula^te said evenness of color distribution in 

8 said video shot. 



1 
2 

3 
4 

1 

2 
3 
4 
5 
6 
7 



1 
2 



70. The method of claim 69, wherein said (b) comprises calculating said 
evenness of color distribution in said video shot as a sum I ^ q-. y.. , wherein 



said Vij are elements of said mai 
singular value decomposition. 



71. A computer-readab 
evenness of color distributions in 
of frames, said method comprising: 

(a) 




said Qj are singular values obtained in said 



containing a program for calculating an 
shot, said video shot comprising a plurality 



performing a singular value decomposition on said plurahty of frames, 
wherein said singula^ value decomposition produces a matrix, each 
column of said matrij^ representing a frame in a refined feature space 
corresponding to a frame in said plurality of frames; 



(b) 



using said matrix to 
said video shot. 



calculate said evenness of color distribution in 



72. The computer readable medium of claim 71, wherein said (b) 



comprises calculating said evenne 



Jraiik(A) ^ ^ 

S (J J Vij ' wherein said 

4 singular values obtained in said si 



s of color distribution in said video shot as a sum 
Vij are elements of said matrix and said ay are 
igular value decomposition. 
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